Content validation of an instrument for medical record audits

ABSTRACT Objective: To build and validate the content of an instrument to conduct medical record audits; to conduct a pre-test. Methods: Methodological study conducted from May/2020 to May/2021 in three stages: 1) development of the instrument by bibliographic survey and benchmarking; 2) content validation using the Delphi technique; 3) application of the instrument and descriptive analysis in a sample of 200 medical records. Results: An instrument was constructed with 11 domains containing sub-items that characterize the quality of care. Two stages of the Delphi technique were necessary to reach a content validity index higher than 0.90. For each domain, a graduated scale with a numerical value from 1 to 4 points was attributed, reflecting the quality of its completion. The average time of application was 35 minutes per record. Conclusions: The tool proved to be viable to support clinical audits to identify the level of excellence and reveal opportunities for improvement in care processes.


INTRODUCTION
Hospital accreditation is a voluntary, constant, and restricted standardization process of evaluating the resources of health institutions to guarantee the excellence of care by means of previously accepted standards (1) .During this process, the patient's medical record is one of the documents that must be examined, the importance of its analysis and audit emerging from the fact it clearly shows the care processes related to patient safety, which are effective in the quality management process (2) .
The patient record is an essential clinical document in the patient's diagnostic and therapeutic pathway.It facilitates the provision and continuity of safe care and promotes structured and effective communication between members of the multidisciplinary team, ensuring the quality of care and supporting process improvement, as well as serving as a means of research (3)(4) .
Thus, medical records are instruments that should be audited to collect and correct waste, irregularities, negligence, and omissions, reflecting the quality of care provided (5) .Moreover, their proper management with accurate and comprehensive record keeping is becoming increasingly crucial for hospital administration and judicial demands, as well as in the evaluation of the institution's level of excellence (6)(7) .
In this context, clinical audits are seen as a continuous improvement program, which involves collecting information from patient records and bedside.It may be prospective and/or retrospective, going through several cyclical steps (5) .As a tool of Clinical Governance, it has the potential to promote change and quality improvements within healthcare organizations, due to its robust method that covers the evaluation of multidisciplinary clinical practice (8)(9) .
According to the NHS (National Healthcare Service) definition, Clinical Governance is a model for continuously improving the quality of services and ensuring excellence in patient care.The seven pillars of Clinical Governance are: Education; Clinical Audit; Clinical Effectiveness; Risk management; Research and development; Patient experience; Transparency (10)(11) .
Thus, the clinical audit is a pillar of Clinical Governance that aims to assess and improve clinical performance in relation to previously established standards, resulting in the refinement of clinical practice (11) .
The clinical audit of medical records makes it possible to identify problems, define evaluation criteria, collect and analyze data, implement changes, and, finally, to redo the audit of the critical process, among other care processes (11) .
Additionally, clinical audits contribute to clinical effectiveness, another pillar of Clinical Governance.Given the current scenario, the focus of health institutions has been to reach a balance between clinical and economic values, that is, to reduce costs and, at the same time, improve care quality and safety (8) .
Therefore, clinical effectiveness is defined as the application of the best science-based knowledge obtained, in addition to the clinical experience of professionals, and patient preferences, aiming to achieve excellence in processes and results (10) .For this, it is necessary to apply the practice of clinical audits to verify the implementation of measures and standards that the institution has established (10) .
By verifying if the assistance provided follows the proper standards and whether the work processes are in conformity with guidelines, the clinical audit allows the development of continuous education with the staff, to guarantee cost reductions regarding the provision of excellent care, bringing no harm to the patient (9) .
As an integral part of the quality management system, which is a coordinated activity aimed at managing processes to achieve excellence in meeting patient demands, clinical audits prioritize the criteria of high volume, high risk, high cost, and when relevant, patient involvement in the evaluation of the care provided, focusing on the development of a cultural environment to improve clinical effectiveness, optimizing health outcomes (11) .It is noteworthy that audit and feedback are recognized as strategic in improving performance and supporting patient safety in healthcare organizations (7) .
One of the priorities of the clinical audit is the Intensive Care Units(ICU), highlighted as units with the highest complexity of structure and care, since they deal with critically ill patients demanding specialized and continuous care; therefore, they have a higher risk of adverse events (12)(13) .Such events occur due to technical, human, and institutional factors and must be prioritized in the analysis, since critically ill patients have particularities that make them more susceptible to fatal events (12)(13) .
Given this, it is essential to perform clinical audits in intensive care units to evaluate the clinical practice of the multidisciplinary team and offer managers and health professionals an analysis of clinical performance, efficiency, and clinical effectiveness of the processes standardized by the healthcare organization (6) .
The lack of measuring tools to help Clinical Audits is its most relevant shortcoming (14) .Developing useful measurement tools to verify the adequacy of knowledge and care processes is a significant step toward ensuring care excellence in health institutions (15) .To this end, it is essential to construct and validate the content of the instrument, so that each assessment item is relevant, comprehensive, and representative of what is to be assessed (15) .Thus, the development and content validation of a tool for conducting clinical audits is essential, facilitating the process of analyzing medical records in healthcare organizations.With it, the quality of processes is improved, and it can also help creating action plans to prevent repeated failures, promoting a culture of patient safety and strengthening the continuous improvement programs of the institution.

OBJECTIVE
To develop and validate the content of an instrument for conducting clinical audits of medical records; to carry out its pre-test.

Ethical aspects
This study was approved by the Research Ethics Committee of the University where it was carried out.

Design, period, and place of study
Methodological study carried out in three phases, from May 2020 to May 2021:

Development of the instrument
To develop the instrument, we considered the criteria proposed by Pasquali: clarity, simplicity, and relevance (16)(17) .The dimensionality of the construct was also taken into account to evaluate the internal structure and semantics of the instrument and define whether it presents a single component, or different and independent study components (18) .
Based on what was described above, the instrument was developed through a bibliographical survey and benchmarking.Benchmarking is a technique that seeks the best practices in the market: it consists in learning from other organizations, seeking more efficient processes and innovative ideas for the institutions to obtain a better performance of their activities (19) .
The benchmarking for the development of the instrument was carried out before the beginning of the study using the experience of its leading researcher, who has five years of experience in Quality Management.This is an area responsible for controlling processes and activities within healthcare institutions.Its main goal is to implement good care practices to improve the results of clinical practice (8) in organizations in the state of São Paulo that have active record review committees and hospital accreditation stamps, conferring high reliability and excellence in the care provided.Moreover, they have essential information for clinical audits.
To build the instrument, we proposed domains and assessment items related to care process excellence and patient safety goals in critically ill patients.These choices were based on theoretical references related to clinical governance, patient safety and intensive care (7)(8)(9)(10)(12)(13) .
The instrument recommends using a retrospective analysis to evaluate the quality of care provided.Thus, it can be applied to all health professionals who are part of the medical record review committee and/or the patient safety center of the institution.

Content validation
For content validation, the Delphi technique was used.We invited professionals to be expert evaluators and assessed their agreement in regard to the composition of the clinical audit instrument for medical records.
The Delphi technique is a systematized method with a broad and enriching approach to capturing ideas and knowledge, helpful in obtaining consensus from specialists on a specific subject through validations articulated in phases (20) .It is an accessible method because it allows the participation of specialists even if they are physically distant (20) .This technique uses consecutive stages to obtain agreement on the content of the instrument.To do so, an e-mail was sent to evaluators, presenting the study and requesting their assessment of the content of the instrument, regarding the clarity and relevance in the wording of each item (20) .
As inclusion criteria in the Delphi technique panel, we considered: having experience in management processes and patient safety, being a member or having been a member of the medical record review committee, and being a researcher with experience in the validation of measuring instruments.Professional experience below six months was the only exclusion criterion.
Specialists were identified at the convenience of the researchers, considering those in a business social network, who had participated in quality assessment and patient safety regulating bodies that met the inclusion criteria for the study.The specialists were contacted through instant messaging application groups and business social media.
The group of experts was composed according to content validation references and the Delphi technique (20)(21) .Seven evaluators who met the inclusion criteria were identified to participate in the assessment of the instrument.The number of specialists varied during the two stages, and two were only present in the first or second stages, leading to a total of nine specialists.The professional category of the group of specialists included higher education professionals (administration and nursing) in the same fields of work: patient safety, management activities, participation in a medical records committee, and quality management in healthcare.
After identification and initial contact via application and social network, the specialists received an e-mail explaining the study's objectives and the importance of participating according to their experience and professional performance.In the e-mail, they were instructed to judge the relevance and pertinence of the instrument and the concepts involved.After they agreed to participate, the questionnaires were sent through Google Forms and analyzed anonymously (21) .
The Delphi technique was performed in two stages to achieve a consensus between the specialists.After we received the instrument from the first stage, a preliminary analysis was performed to determine its composition and the validity of its items.Based on this analysis, a second version was elaborated, incorporating the modifications suggested by the specialists in the first stage.A new e-mail was sent to all those who participated then, including the new version, a summary of the previous stage, and instructions for judging the content to avoid deviating from the main points of the instrument.

Data collection procedure
The data collection procedure took place in a small private institution in the interior of the state of São Paulo, with demands for clinical and surgical patients, in the Cardiology Intensive Care Unit.
This unit has 12 inpatient beds, and its priority care is for patients in the postoperative period of interventional procedures performed in the hemodynamics unit.The main specialties attended at the institution are cardiovascular, neurology, arrhythmology, and orthopedics (the latter, performed in the Surgical Center).In addition, the institution is a reference for clinical patients from other institutions in the region.
The pre-test was performed based on 200 medical records from 2020 and 2021.The choice of these records followed the de

Content validation of an instrument for medical record audits
Simplicio C, Dini AP.
eligibility criteria: patients with a length of stay > 3 days; patients who have undergone different lines of care (Emergency; Intensive Care Unit; Surgical Center; Inpatient Units); patients ≥ 60 years with multiple comorbidities; and patients elected for guideline-based management (sepsis, chest pain, stroke, and/or palliative care).Due to the pandemic period in which the pre-test of the instrument was conducted, the medical records made available by the Medical Records Committee of the institution were analyzed by convenience.The sample size calculation followed the recommendation of a statistical professional, who oriented to use between five and ten evaluations per item.A term of responsibility was sent to the institution for appreciation and authorization as to the pre-testing of the instrument, which was built to consult the medical records.The instrument was applied by the lead researcher of the study, a member of the medical records review committee of the institution under study.The database is available through the following link: https://doi.org/10.25824/redu/PTAPUA

Analysis of the results and statistics
Besides evaluating the agreement on the domains and items of the instrument, the Content Validity Index (CVI) was evaluated, in which the specialists attributed to each item one of the four options: 1.Not relevant or not clear for medical record audits; 2. Needs major revisions to be relevant or clear for record audits; 3. Needs minor revisions to be relevant or clear for record audits; 4. Relevant and representative for record audits.
To calculate the CVI for each item, the total number of "3" or "4" options assigned by the specialists was considered, divided by the total number of answers; items with a CVI higher than 0.90 were considered valid (21) .
For the pre-test analysis we used descriptive statistics, which is a means of organizing and summarizing the main characteristics observed in a set of data, allowing the researcher a better understanding of the data studied (22) .

RESULTS
The initial development of the instrument was composed of 11 domains, each with sub-items that characterized the quality of care provided by the multidisciplinary team.
To reach a content validity index above 0.90 in all sub-items of the domains, two stages of the Delphi technique were required, as shown in Table 1.In the first stage, the judges gave suggestions for more clarity in the domains "Device-Related Infections," "Managed Care Risks," "Multidisciplinary Team Actions," and "Safety Culture." Based on the suggestions, a new version of the domains and items was presented in the second phase of the Delphi technique, for content validation.
After content validation and the final version of the "Clinical Audit of Medical Records" instrument, 200 medical records were audited.The assessment time varied between 30 and 40 minutes per record.The pre-test of the instrument is presented, per domain, in Tables 2, 3, and 4.
Table 2 presents data from the analysis of medical records related to Patient Identification, Effective Communication, Safety in Medication Prescription, Use, and Administration.Table 3 shows the data extracted regarding Device-Related Infection, Care Risks, Guideline-Based Protocols, and Medical Procedures.Table 4 shows the data extracted using the instrument to analyze medical records related to Medical Procedures, Nursing Procedures, Multidisciplinary Team Actions, and Safety Culture.

DISCUSSION
The development and content validation of an instrument for clinical auditing was a challenging trajectory in which we sought to contribute to clinical and scientific practice by providing an innovative instrument that could make tangible and measurable the quality/safety registered in medical records.To do so, it was necessary to rely on references about the development and validation of instruments (1)(2)15,21) , and on the knowledge and experience of professionals involved in the auditing of medical records and quality management in healthcare.
In the process of building the instrument, benchmarking was also a strategy to allow the application of the instrument in different public and private institutions, since benchmarking enables continuous improvement, aiming to enhance the performance of the institutions' critical processes (19) .
Among the criteria necessary to build a good measuring instrument, we emphasized objectivity and clarity, i.e., long sentences with full details can provide a tiring and unfocused reading (23) .Therefore, taking into account the clinical audit process to standardize the evaluation of medical records and make it more objective, the specialists' considerations in the two phases of the Delphi technique enabled reformulations in the domains and items of the validated instrument, in order to analyze the conformity of the information registered in medical records (20) .Content validation is the first stage in the development of measurement tools, and having achieved a CVI higher than 0.9 for all items of the tool indicates an excellent level of content validity regarding the clarity, readability, relevance, and pertinence of each item assessed (15) .
The process of content validation also has associations with the scope of the instrument (16)(17)20) . Thus the application of the instrument should be prioritized in the clinical audit of the medical records of chronically ill or critically ill patients, due to the composition of the items built and to the potential it presents for the management of processes, and of financial and human resources inherent to the complexity of care for this profile of patient (24)(25)(26) .
In the pre-test of the instrument, after the content validation was completed, the average application time of 35 minutes per medical record showed that the application of the tool is feasible in the clinical audit process (7,(15)(16)(17)(18)(19)(20) .
The process of validating an instrument should be continuous, so construct evaluation, validity, reliability, homogeneity, and equivalency should be continued in the institutions that adopt it (15) .This study had important limitations, such as the impossibility to perform inter-evaluator reliability.This limitation was due to the fact that data collection took place during the social distancing period enforced to contain the spread of SARS-CoV-2, reason why the Ethics Committee did not approve the initial study protocol for simultaneous data collection by two researchers.
Additionally, the application of the instrument in different institutions was not authorized due to the pandemic that took place in the period of data collection.This prevented the performance of exploratory factor and reliability analyses of the internal consistency using Cronbach's alpha (27) .
It is noteworthy that, in the pre-test, it was possible to experience the importance of clinical auditing in the analyses of care processes, enabling the discrimination of elective processes for improvement of consolidated processes, i.e., the tool was perceived to be adequate for the assessment of the maturity of care processes.
Therefore, the application of the instrument validated in this study is recommended to standardize the auditing of medical records, so hospitals can establish a method for continuous improvement and diagnose which are their best-defined, mature, and organized processes, aiming at excellence in care.Thus, its application may bring many benefits, from the perception of quality and safety by the multidisciplinary team to the process of hospital accreditation, which presupposes the implementation of good care practices based on quality management and structural changes (28)(29)(30) .
The aggregation of theoretical references, knowledge of professional specialists, benchmarking in the development, and validation of the content of the clinical audit instrument built in this study allowed us to make available a tool for hospital institutions to standardize the evaluation of medical records and thus analyze interdisciplinary processes and practices that ensure the quality of care in a manner committed to continuous improvement.

Study limitations
As a limitation of this study, it should be noted that due to the pandemic during the period of data collection, the interevaluator reliability analysis of the clinical audit instrument was not authorized, despite being recommended before its implementation.

Contributions for the field of nursing, health, or public policy
The development and content validation of an instrument for clinical audits will optimize the management of the work process regarding patient safety and quality of care for critically ill patients, since these clients require complex care and are exposed to many significant care risks.Therefore, having a tool that guides the active search for processes that need to be improved is of great importance in the work process of quality management and for patient safety, which together have a single goal: to develop good practices to provide excellent care, free of harm.
The clinical audit, through the instrument validated in this study, makes it possible to base training and action plans on the processes that need to be improved and, thus, provides a program for continuous improvement within health institutions, aiming at the reduction of avoidable adverse events, at focused care, and at a good patient experience.

CONCLUSION
The study allowed us to build and validate the content of an instrument for conducting clinical audits of medical records.All items presented a CVI higher than 0.9, which denotes an adequate content validation.The participation of specialists involved in quality management in health care in the process of content validation enhanced the composition of the clinical audit instrument in 11 domains and 52 evaluation items.
The pre-test of the clinical audit instrument in 200 medical records with an average application time of 35 minutes per record shows the feasibility of this tool in daily work.
Therefore, the use of the instrument is recommended to evaluate the maturity of care processes and to establish continuous improvement programs in healthcare institutions.

ACKNOWLEDGMENT
The Programa de Pós Graduação em Enfermagem da Universidade Estadual de Campinas (Postgraduate Program in Nursing at the State University of Campinas).

CONTRIBUITIONS
Simplício C e Dini AP contributed to the conception or design of the study/research, analysis and/or interpretation of data, final revision with critical and intellectual participation in the manuscript.

Table 4 -
Pre-testing of the instrument, analysis of medical records regarding Medical Procedures, Nursing Procedures, Multidisciplinary Team Actions, and Safety Culture, Campinas, São Paulo, Brazil, 2021

Table 2 -
Pre-testing of the instrument and analysis of medical records regarding Patient Identification, Effective Communication, Safety in Medication Prescription, Use, and Administration, and Safer Surgery, Campinas, São Paulo, Brazil, 2021 Caption: * Lack of information; + Inaccurate and/or incomplete information; ǂ Clear but incomplete information; § Accurate and complete information; ǁ Not applicable.

Table 3 -
Pre-testing of the instrument, analysis of medical records regarding Device-Related Infection Prevention Measures, Managed Care Risks, Managed Protocols, Campinas, São Paulo, Brazil, 2021Caption: * Lack of information; + Inaccurate and/or incomplete information; ǂ Clear but incomplete information; § Accurate and complete information; ǁ Not applicable.
Caption: * Lack of information; + Inaccurate and/or incomplete information; ǂ Clear but incomplete information; § Accurate and complete information; ǁ Not applicable.